1. Field of the Invention
The present invention relates to estimating a mental decision to activate a task related function which is selected by a visual cue and, therefore, can be used to control machines from a visual display by eye gaze alone.
The present invention has many potential applications in the medical, scientific, engineering, manufacturing, military, entertainment, and other fields. The present invention may be used as a tool for medical diagnosis of ocular functions, as an aid to the paraplegic handicapped, as an instrument for providing measurement of ocular functions and workload in human factors studies, as a measure of subject training, as a tool for fatigue monitoring, as part of an electronic safety net to detect performance degradation due to pilot incapacitation in piloted and teleoperated vehicles, as a component of an electronic intelligent pilot-vehicle interface used for situation awareness aiding in piloted and teleoperated vehicles, as a tool for task scan analysis including situation awareness measuring, as a controller of machines and computer games, and for advertisement and usability analysis.
Particularly, the present invention is utilized to control computerized machines from an electronic video display by the ocular gaze point of regard alone. Examples of machine control by ocular functions are: (1) updating computer generated information displays, (2) selecting panel switches and instruments, (3) controlling the fidelity of computer generated imagery scene inserts in simulations, (4) controlling the viewing direction of remotely located cameras, (5) controlling the movement of teleoperated robotics platforms or vehicles, (6) selecting display subareas for automated scene analysis in aided target recognition, (7) designating targets from direct sight or from a sensor display, and (8) weapon system pointing.
The present invention has particular applications to time shared concurrent tasks where hands are involved in a continual time critical pilotage task and eyes are used intermittently to control a discrete task. The present invention enables both tasks to share a common visual working area with overlaid visual images. Therefore, the present invention allows task interference to be reduced by dedicating eye-movements and visual attention to the same working surface.
An example of such an application would be single pilot nap-of-earth low-level helicopter flight while updating on-board heads-up displays. A similar application is teleoperations of remote vehicles from video displays with camera control. Another such application is to the operation of completely enclosed armored vehicles with "transparent" or "see through" armor. There the operator would see a video projection of the outside scene, recorded by externally mounted cameras and relayed to internal monitors. The operator would use the present invention to control displays overlaid on the scene projection while concurrently performing the vehicle pilotage task. Similar comments apply to the piloting of "glass cockpit" designs for completely enclosed, high performance aircraft.
The present invention with a properly designed oculometer can also be used with head-mounted video displays in many application fields. The head-mounted video displays, such as those developed for virtual reality, stereographic displays, monocular or binocular vision helmet mounted displays, and night vision goggles used in piloted helicopters, vehicles, and teleoperated robotics control stations are all suitable.
2. Description of the Background Art
The conventional method of decision estimation for control of machines by ocular functions is based on measuring the eye gaze fixation duration, which commonly is longer for a visual cue of task interest than for a visual cue of disinterest. However, the statistical distributions of the interest and disinterest fixation durations tend to overlap and are clearly not separable. In practice, the user must extend the gaze duration with an unnaturally directed stare to designate a cue of interest. Usually, the user follows the display response to the cue selection with a motor action like a button push to confirm the visual selection. In some designs the user must execute an extended stare for cue selection in conjunction with the motor action to indicate a selection. This need for an extensively forced gaze tends to interrupt the task flow since any naturally occurring visual search patterns are momentarily suppressed. It can also increase ocular fatigue due to the corresponding reduction in eye blinks and the associated reductions in corneal lubrication and oxygenation. Furthermore, the need for a confirming motor action increases the workload of the viewer.